.« 



M&G No. 50019.0241US01 / P05641 

ADAPTIVE SETTING OF WAVELET SUB-BAND THRESHOLDS FOR LOW- 
LIGHT IMAGE DENOISING 



Field of the Invention 

5 The present invention relates to denoising, and, in particular, to a method 

and apparatus for selecting a threshold for each sub-band for wavelet-based denoising. 

Background of the Invention 
A typical film-type camera includes a mechanical shutter. Light is 
reflected off of a scene from either artificial or natural light conditions. The camera 

10 receives the reflected light when the mechanical shutter is opened. While the shutter is 
opened, the reflected Ught is focused on a film that contains hght sensitive components. 
The light sensitive components of the film create a chemical recording of the received 
hght. The image, or scene, that is recorded on the film corresponds to the total amount 
of light that directed onto the film during exposure to the reflected light. The total 

15 amount of light directed onto the film is related to the aperture of the shutter, and the 
exposure time. The exposure time of the film corresponds to the length of time that the 
mechanical shutter is open. The exposure time is a fixed time interval that is manually 
adjusted by an operator. 

Digital cameras are similar to film-type cameras in that light is reflected 

20 off of the scene and received by a light sensitive component when the shutter is open. 
The light sensitive component may be a charge coupled device (CCD), or a 
complementary metal oxide semiconductor (CMOS) device. The image sensors convert 
light into electronic signals when exposed. The signals are provided to an image 
processor to store the image of the recorded scene. Li digital cameras, the shutter may 

25 be a mechanical shutter that is similar to that found in the fihn-type camera, or an 
electronic shutter mechanism. 

A typical image sensor includes an array of image capturing points, or 
pixels. Each pixel is arranged to provide a signal that is responsive to light. The signal 
level that is provided by each pixel is related to the intensity of the light that is received, 
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and the length of time that the pixel is exposed. A CMOS array is arranged in rows and 
columns that may be read individually, while a CCD array typically provides a serial 
read out of the electronic signals. The electronic signals are typically converted into 
digital signals that are stored in a memory circuit, such that the digitized image is 
5 captured. The digitized image can be digitally manipulated for color adjustment, image 
contrast, image scaling, and image rotation functions. 

Brief Description of the Drawings 
Non-limiting and non-exhaustive embodiments of the present invention 
are described with reference to the following drawings. 

FIG. 1 illustrates an image processing system; 
FIG 2A through FIG-2C illustrate a process for detemiining threshold 

settings; 

FIG. 3 illustrates example system functions for wavelet thresholding; and 
FIG. 4 illustrates a process for providing a denoised image, in 
accordance with aspects of the present invention. 

Detailed Description of the Preferred Embodiment 

Various embodiments of the present invention will be described in detail 
with reference to the drawings, where like reference nmnerals represent like parts and 
assemblies throughout the several views. Reference to various embodiments does not 
20 limit the scope of the invention, which is limited only by the scope of the claims 
attached hereto. Additionally, any examples set forth in this specification are not 
intended to be limiting and merely set forth some of the many possible embodiments for 
the claimed invention. 

Throughout the specification and claims, the following terms take at 
25 least the meanings explicitly associated herein, unless the context clearly dictates 
otherwise. The meanings identified below are not intended to limit the terms, but 
merely provide illustrative examples for the terms. The meaning of "a," "an," and "the" 
includes plural reference, the meaning of "in" includes "in" and "on." The term 
"connected" means a direct electrical connection between the items connected, without 
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any intermediate devices. The term "coupled" means either a direct electrical 
connection between the items connected, or an indirect connection through one or more 
passive or active intermediary devices. The term "circuit" means either a single 
component or a multiplicity of components, either active and/or passive, that are 
5 coupled together to provide a desired function. The term "signal" means at least one 
current, voltage, charge, temperature, data, or other signal. 

Briefly stated, the invention is related to a wavelet-based denoising 
system and process. A corrected reference image is acquired and stored. A test image 
is adjusted using a selected gain setting such that the mean pixel value of the adjusted 

10 test image matches the mean pixel value of the reference image. The adjusted image is 
decomposed into sub-bands using a wavelet transformation. Thresholding is applied to 
a selected sub-band with a selected threshold setting. An inverse wavelet 
transformation is then applied to the sub-bands to provide a denoised image. This 
process is repeated such that each sub-band is tested with multiple threshold settings. 

15 The denoised images are analyzed to locate a threshold setting that is approximately 
optimal for each sub-band. This process is repeated for muhiple light settings. A gain- 
threshold map is constructed by mapping each selected gain setting to the 
approximately optimal threshold settings for each sub-band at the selected gain setting. 

The process for constructing the gain-threshold map may be performed 

20 using an image processing system. The gain-threshold map may be used in a process 
for image denoising. The process for image denoising may also be performed using an 
image processing system. 

Example System 

FIG. 1 illustrates an image processing system (100) that is arranged in 

25 accordance with aspects of the present invention. System 100 comprises an image 

sensor (102), a row control logic block (104), a column control logic block (106), a 

column amplifier block (108), a front end processing block (110), an image processing 

and control block (112), and a shutter (114). Image processing and control block 112 

includes an analog-to-digital conversion (A/D) block (116), a processing block (118), 

30 and a memory (120). Programs (122) may be embodied in memory 120. 
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Image sensor 102 comprises a plurality of pixels that are organized into 
rows and columns. The pixels are responsive to incident light when shutter 1 14 is open. 
Shutter 114 is responsive to a shutter control signal (SCTL). Row control logic block 
104 is configured to select rows in response to a row control signal (RCTL). Column 
5 control logic block 106 is configured to select columns in response to a column control 
signal (CCTL), Column amplifier block 108 is configured to adjust the output signals 
fi*om selected pixels to provide a raw analog signal (Al). Front end processing block 
110 is configured to process signal Al to provide a processed analog signal (A2). In 
one example, fi-ont end processing block 110 is arranged to provide offset and gain 
10 correction for the raw analog signal (Al). In another example, front end processing 
block 1 10 is a correlated double sampling circuit (CDS) that provides offset and/or gain 
correction. 

Image processing and control block 112 is configured to provide signals 
SCT, RCTL, CCTL, and a digital image signal (OUT). Image processing and control 

15 block 112 is responsive to signal A2. A/D block 116 is arranged to convert signal A2 
into a digital signal (Dig) in response to signal A2. A/D block 116 is controlled by 
various timing signals (Timing) that may be provided by processing block 118. 
Processing block 118 is arranged to process the digital signal (Dig) to perform image 
denoising. Processing block 118 is arranged to communicate with memory 120, which 

20 receives the processed digital signal. 

In one example, programs 122 are stored in memory 120, where the 
programs are employed by processing block 118 to perform wavelet-based image 
denoising in accordance with aspects of the present invention. In another example, the 
processing block (118) and memory block (120) are included in a single functional 

25 block such as a specially designed integrated circuit (e.g., an ASIC). In still another 
example, the functions provided in the processing block (118) are provided by a 
microprocessor, a microcontroller, or some other general purpose processing unit. In 
yet another example, the functions provided by the processing block (1 18) are provided 
by an electronic circuit that is specially configured such as a programmable logic device 
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(PLD), a programmable gate array (PGA), a programmable logic array (PLA), as well 
as any other appropriate device or devices. 

Example Process for Constructing a Gain-Threshold Map 

FIGS 2A-2C illustrate a process (200) for determining threshold settings 
5 in accordance with aspects of the present invention. Processing begins at start block 
202, and proceeds to block 204. Processing proceeds from block 204 to block 216. 
According to one example, block 204 comprises blocks 206, 208, 210, and 212. At 
block 206, a reference image is acquired using good lighting conditions. According to 
one example, the shutter (114) is opened. Light is reflected off of the scene and 

10 received by image sensor 102. Column ampHfier block 108 ampUfies the output signals 
from selected pixels to provide a raw analog signal (Al). Front end processing block 
110 processes signal Al to provide a processed analog signal (A2). A/D block 116 is 
arranged to convert signal A2 into a digital signal (Dig) in response to signal A2. The 
digital image (Dig) is stored in memory 120. 

15 Processing then proceeds from block 206 to block 208. At block 208, a 

dark image is acquired (e.g., from system 100). Processing then proceeds from block 
208 to block 210. At block 210, the locations of the bad pixels identified from the dark 
image are stored in memory. According to one example, bad pixel correction may be 
performed by processing block 1 18 under the control of a program 122. According to 

20 one example, each good pixel location has a corresponding value of approximately zero 
when exposed by a dark (e.g., black) image. A pixel value greater than approximately 
zero corresponds to a bad pixel value. In one example, a bad pixel value may result 
from speckle noise that is caused by leakage current in the pixel. Higher non-zero pixel 
values indicate worse pixels. A predetermined number (N) of bad pixels are stored in a 

25 memory to map the bad pixel locations. According to one example, N corresponds to 
1024. According to another example, the bad pixel locations are ordered from worst to 
best. Processing continues from block 210 to block 212. 

At block 212, the reference image that is stored in memory is corrected 
for bad pixels. For example, the values associated with bad pixel locations (as 
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determined at block 210) may be replaced by the average of the values associated with 
the good neighboring pixels. Processing then proceeds from block 212 to block 216. 

At block 216, a test image is acquired at a newly selected light setting. 
The test image should be matched to the reference image such that the only difference 
5 between the reference image and the test image is the light setting. Processing flows 
from block 216 to block 218. At block 218, the test image is corrected for bad pixels 
such that the values associated with the bad pixel locations (as determined at block 210) 
are replaced. As described previously, the values associated with bad pixel locations 
may be replaced by the average value of the good neighboring pixels. Processing then 

10 proceeds from block 218 to block 220. 

At block 220, the corrected test image is adjusted by amplifying the 
value associated with the pixels according to a gain setting. The gain setting is related 
to the selected light setting. According to one example, the gain setting is selected such 
that the mean pixel value of the adjusted test image matches the mean pixel value of the 

1 5 reference image. In another example, the maximum pixel values may be matched rather 
than the mean pixel values. According to one example, adjusting the corrected test 
image is accomplished by processing block 118 under the control of a program 122. 
Processing proceeds from block 220 to block 222. 

At block 222, a wavelet transformation is applied to the adjusted image 

20 such that the adjusted image is decomposed into a plurality of sub-bands. According to 
one example, a three-level Haar wavelet transform is performed on the adjusted image. 
According to one example, the wavelet transformation is performed by processing block 
118 under the control of programs 122. Processing proceeds from block 222 to block 
226. 

25 At block 226, one of the sub-bands is selected. Processing continues 

from block 226 to block 228. At block 228, approximately optimal threshold settings 
for the selected sub-band are determined. Processing flows from block 228 to decision 
block 240. 

At decision block 240, the system determines whether all of the sub- 
30 bands have been tested for the selected test image. Processing proceeds from decision 
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block 240 to block 226 when additional sub-bands should be tested. Alternatively, 
processing flows from decision block 240 to decision block 242 when all of the sub- 
bands have been tested. At decision block 242, the system deteraiined whether all of 
the light settings have been tested. Processing continues from decision block 242 to 
5 block 216 when additional lights settings need to be tested. Alternatively, processing 
flows from block 242 to end block 244 when all of the light settings have been tested. 

According to one example, block 228 comprises blocks 230, 232, 234, 
238, and decision block 236. At block 230, threshold parameters are selected for the 
selected sub-band. According to one example, the threshold parameters are selected 

10 according to a threshold percentage (P%). A threshold value is selected such that P% of 
the wavelet coefficients have a magnitude that is less than the threshold value. The 
coefficients may be stored in histogram bins for thresholding when threshold 
percentages are utilized. At block 232, thresholding is applied to the selected sub-band 
using the selected threshold parameters. Processing continues from block 232 to block 

15 234 where an inverse wavelet transformation is performed on each of the sub-bands to 
provide a denoised test image. According to one example, the denoised test image is 
output to a display for viewing by an operator. Processing continues from block 234 to 
decision block 236. 

At decision block 236, the system determines whether all of the 

20 threshold parameters have been tested for the selected sub-band. Processing flows from 
decision block 236 to block 230 when additional threshold parameters should be tested. 
Altematively, processing proceeds from decision block 236 to block 238 when all of the 
threshold parameters have been tested. At block 238, approximately optimal threshold 
settings are determined for the selected sub-band. For example, an operator may view 

25 each of the denoised test images that were tested for the current sub-band. The operator 
may select one of the denoised test images as the subjectively best image. The 
threshold settings used for the subjectively best image may be selected as the 
approximately optimal threshold settings for the selected sub-band. 

Each of the threshold parameter sets is tested in block 228. According to 

30 one example, for each sub-band, a threshold percentage of 5% is first selected at block 
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230. The threshold percentage is incremented by 5% each time block 230 is reached 
iintil a new sub-band is selected. At block 238, the system evaluates the results and 
determines which threshold percentage provided an approximately optimal denoising of 
the selected sub-band. 

5 The results of process 200 may be used to construct a gain-threshold map 

in memory 120. The gain-threshold map may be organized as a lookup table. The 
lookup table links each of a plurality of gain settings to the associated set of threshold 
parameters for each sub-band for the gain setting (as determined by process 200). 
According to one example, the threshold parameters correspond to threshold 
10 percentages. 

According to one example, process 200 may process the entire image 
such that the entire image is stored in memory. According to another example, process 
200 may process a part of the image such that memory requirements are reduced. For 
example, process 200 may be applied to two lines of data at a time. The quality of the 
15 denoised image is degraded when the number of lines of data is reduced. 

Process 200 may be applied to color images in addition to grey-scale 
images. Each color channel is processed separately according to process 200. White 
balancing is performed on the reference image such that each color channel may have a 
different gain setting. 

20 Process 200 may be used to test threshold percentages for each sub-band 

at each of a plurality of gain settings. Process 200 may also be used to optimize other 
parameters such as wavelet filter type and length, the number of decomposition levels, 
and the thesholding type. For example, other types of wavelet types may be tested, such 
as Daubechies wavelet transformations and Symmlet wavelet transformations. Haar 

25 wavelet transformations may be preferable under typical operating conditions. 
Daubechies wavlet transformations may be preferable in applications where a blocky 
output image is undesirable and the original noise level is high. 

Example of Wavelet Thresholding 

FIG. 3 illustrates example system functions (300) for wavelet 

30 thresholding that is arranged in accordance with aspects of the present invention. 
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System 300 comprises a forward wavelet transformation block (302), a thresholding 
block (304), and an inverse wavelet transformation block (306). According to one 
example, system 300 is included in block 118 and is controlled by programs 122. 

Forward wavelet transformation block 302 is configured to perform a 
wavelet transformation on an input signal (x[n]). According to one example, wavelet 
transformation block 302 is configured to perform a three level Haar wavelet 
transfomiation on x[n] to provide ten sub-bands as follows. At each level, an input 
signal is filtered by low pass and high pass decomposition filters to provide filtered 
outputs. The filtered ou^uts are sub-sampled by two (i.e. every other sample is 
discarded). Each sub-band can be determined by the equation: z[n] = 
^x[n]»h[2k-n], where z[n] is the sub-band obtained after filtering by a 

n 

decomposition filter with a frequency response of h[n] and then subsampling the filter 
output by two. Low and high frequency bands are provided in the horizontal direction 
and low and high fi-equency bands are provided in the vertical direction such that four 
resulting fi-equency sub-bands (low-low, low-high, high-low, and high-high) are 
provided. The low-low sub-band is the input signal to each subsequent level, because 
most of the image data is concentrated in the low-low sub-band. According to one 
example, forward wavelet transformation block 302 may be used in block 222. 

Threshold block 304 is arranged to threshold the decomposition output 
coefficients. Coefficients having a magnitude that is less than the threshold level are set 
to a value of zero. The coefficients having a magnitude that is greater than the threshold 
value are not changed when hard thresholding is implemented. The threshold value is 
added to coefficients that are less than zero and have a magnitude that is less than the 
threshold value when soft thresholding is implemented. The threshold value is 
subtracted from coefficients that are greater than the threshold when soft thresholding is 
implemented. According to one example, threshold block 304 may be used in block 
232. 

Inverse wavelet transformation block 306 is configured to perform an 
inverse wavelet transformation on the sub-bands to provide an output image signal 
(y[n]). Each threshold output is filtered with reconstruction wavelet filters. According 
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to one example, each filtered output is then upsampled by two, first through rows and 
then through columns. According to one example, inverse wavelet transformation 
block 306 may be used in block 234. 

Example of Image Denoising Using Gain-Threshold Map 

5 FIG. 4 illustrates a process (400) for providing a denoised image, in 

accordance with aspects of the present invention. Processing begins at a start block 
(402). Processing then proceeds from start block 402 to block 404. At block 404, an 
image is captured and stored in memory. According to one example, the shutter (1 14) 
is opened. Light is reflected off of the scene and received by image sensor 102. 
10 Column amplifier block 108 amplifies the output signals firom selected pixels to provide 
a raw analog signal (Al). Front end processing block 110 processes signal Al to 
provide a processed analog signal (A2). A/D block 1 16 is arranged to convert signal A2 
into a digital signal (Dig) in response to signal A2. The digital image (Dig) is stored in 
memory 120. 

15 Processing then proceeds from block 404 to block 405, where the 

captured image is processed. For example, the capture image may be processed in 
processing block 118 under the control of a program (122). Processing proceeds from 
block 405 to block 418. At block 418, the denoised image is stored or output. 
Processing continues from block 418 to block 420 where processing is terminated. 

20 According to one example, block 405 comprises block 406, block 408, 

block 410, block 412, block 414, and block 416. At block 406, the captured image is 
corrected for bad pixels. According to one example, correction of the bad pixels is 
performed by processing block 118 under the control of a program 122. Processing 
then proceeds from block 406 to block 408. At block 408, the corrected image is 

25 adjusted by amplifying the pixel values according to a gain setting. The gain setting 

may be selected manually by a user, or automatically according to the lighting 

conditions. Processing proceeds from block 408 to block 410. At block 410, threshold 

settings are retrieved from a gain-threshold map according to the selected gain setting. 

The gain-threshold map may be provided as a lookup table that links each of a plurality 

30 of gain settings to associated threshold parameters for each sub-band for the selected 
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gain setting. According to one preferred embodiment, the optimal set of threshold 
parameters comprises an optimal threshold percentage for each sub-band for the 
selected gain setting. The threshold percentage corresponds to a threshold value. The 
threshold percentage represents the percentage of wavelet coefficients that have a 
5 magnitude that is less than the threshold value. 

Processing flows from block 410 to block 412. At block 412, a wavelet 
transformation is applied to the adjusted image to provide sub-bands. According to one 
example, the wavelet transformation is performed by wavelet transformation block 302. 
Processing then proceeds from block 412 to block 414, where thresholding is appHed to 

10 the sub-bands. According to one example, thresholding is performed by threshold block 
304. The thresholding is applied according to the threshold parameters that were 
retrieved from the gain-threshold map for the selected gain setting. According to one 
example, soft thresholding is applied. Processing then proceeds form block 414 to 
block 416. At block 416, an inverse wavelet transformation is applied to each of the 

15 sub-bands to provide the denoised image. According to one example, the inverse 
wavelet transformation is accomplished by block 306. Processing then proceeds from 
block 416 to block 418. 

The gain-threshold map may be constructed from results obtained from 
applying process 200. Different threshold values for each sub-band are preferable when 

20 the gain setting is high and also when the noise level is high. The threshold settings 
may be preferably set to the median value of wavelet composition when noise levels are 
low. 

Process 400 may be performed on color images and grey-scale images. 
Each color channel is processed separately according to process 400. Each color 
25 channel is adjusted differently for white balancing. The settings for each color channel 
are retrieved separately on the gain-threshold map since the gain amount is different for 
each channel. 

The above specification, examples and data provide a complete 
description of the manufacture and use of the composition of the invention. Since many 
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embodiments of the invention can be made without departing from the spirit and scope 
of the invention, the invention resides in the claims hereinafter appended. 
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